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ABSTRACT ^ ot perceptlon dttr i ng reading, the use of 

*ve movement contingent control of the stimulus display. has" proved to 
be a useful research technique. With such a system, it is .Possible to 
eLtrimentalirmanipulate, in real time, the characteristics of thy 
StfSulu!" display thlt is present on selected fixations as reading is 
in SroSJels and to observe the effects of the manipulation on the eye 
movement pattern. The technique has been used in a situation where 
^rSisPlSy must be .changed during the period of a saccadic eye 
m«5«mant and when it was necessary to make a change at some tinje, 
SSSTtht beginning of a fixation! Among the instrumentation concerns 
\?hat are implrtSnt to consider when setting up a system to do this 
tvoe It researcTare. the -following: ( 1) the eye movement equipment 
musThave fast throughput and low-noise and y «^« ew y ^^^ n 
with hioh frequency: (20 the program must sample the eye position 
Jith a high f?SqueXcy detect saScade onset* and make the decision 
Hint whether to initiate a display change as early as the study 
p^ts, SSd miSim5ze tV amount of computation involved ^ changing 
the imaae: and (3) the display device must be capable of being . 
rapidlj refreshed and- of initiating ^presentation of -a new image 
part way through the refresh cycle. (PL) 
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Abstract , ' 

in the study of perception during reading the use of eye movement contingent 
control of , the .stimulus display has >oved to b_e a useful research technique. 
With such a system it is possible to Experimentally manipulate, in real tike, 
the characteristics of the stimulus display that is present on selected 
fixations as reading is in progress,. and to observe the effects of the 
manipulation on the eye movement pattern. This technique can also be used in 
the study of other on-going, visually based tasks.. This paper provides 
examples, of how the 'technique has been used, and describes a number of the 
instrumentation concerns which are important to consider, when setting up a 

„System-to-5do-this-type-bf— research^-7 — ; ~ • < >. 
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Instrumentation Considerations in Research Involving - : 
Eye-movement Contingent- Stimulus .Control 

There are three possible reasons for monitoring eye movements in 
psychological research. First, the eye movement records can serve as a source 
Of data. Sometimes aspects of these data, such as the durations of fixations 
or the lengths or directions or velocity . patterns 'of the eye movements ; are 
the dependent variables in the study. At other times, the data are used to 
determine which trials should- be excluded^ from analysis. Second, real time 
information about the eye movements can be ueed as a b, as is for making on-line 
experimental- manipulations in' the -stimulus. For example,, a tone can sound or 
a change f can be- made in the visual array when the eyes fixate a certain 
location/ Third, real t^ime information about' the eye movements can be used as 
a basis for other forms of 'measurement. EMG recording can be enabled only 
when the. eyes fixate a certain location, or EEQ data <can be selected and 
averaged based on the location and/or movements of. the eyes. <^ 

The .focus of the present %v paper will be on the second of these 
alternatives: usinge^ejovemej^oj^rl^ for jnaklng stimulus 



manipulations. For a number of years we have been studying the nature of the 
i perceptual and eye movement control processes taking place as people read. 
This "required the development of a computer-based system that was capable of,. . 
monitoring people's eye movements as they read r and, on the basis of that 
information, making real time, changes in the text from which they were' reading 
(McConkie, Zola, Wolverton, 4 Burns, 1978) . Using this system, we are able to 
—allow subjects to read text displayed on a cathode-ray tube (CRT), and^as they 
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arvs reading, to manipulate "the stimulus pattern that is present in the region 
■• ► * * * • ~ * 

where they look for a particular fixation, or at a particular time during the 

fixation. We then examine the effect which these display manipulations 

•♦»... ' • . , . 

produce on the eye movement patterns as a way of learning about the processes 
being studied* . 

These techniques have been extremely fruitful *in investigating aspects of 
the perceptual processes as they occur during reading (McConkie, 1983; Rayner, 
1983). They have also been used- to study perception in simpler tasks 
(Bridgeman, Hendry, 4 Stark, 1975? fruitf,' Yahtis/'Wohides, 1983? Leyy-Schoen 
4 Rigaut-Renard v4 1981). We beUeve that theV could be equally useful in the ^ 
study of perception in other visual lyrbased tasks, such as visual search and 
picture perception. Anticipating that other researchers are likely to attempt 
to develop research of. this sort, . tfce purpose of the present paper is to 
describe some of the considerations that must *be talten into account in 
selecting, equipment for this type of research. While it will .deal primarily \ 
with eye movement contingent control of visual displays, many of the points, 
made will also be relevant to on-line control of other- forms of stimulus 
manipulation or -data collection. ^ ' . -> 

In ord$r to illustrate somejof the perceptual 'issues whiclvcaft be 

investigated using these, techniques, we will briefly describe four examples ' 

• p - '"? . * 

from our own research. ■ *■ ■ 

-^.^*^ ra _l^ter^ 

: centered at the exact' location to ,which they are sent on a saccade in ~ 

\ . . 

order for processing to 'proceed normally. This was investigated by 
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causing, certain fixations to be "misplaced" slightly as people were 
reading.' During certain saccades, the text was shifted two character 
positions left or right on the CRT, so that when the eyes stopped for the • 
next fixation they were centered at a. location different from Where they 
normally would have been (McConkie, Zola, 4 Wolverton, Note 1; see. also 
O 1 Regan, 1981). . ; ' 

!.n order to determine whether visual information is acquired and used / 
from specific visual r^gio'ns during fixations in .reading, the letters in ^ 
t£ese regiohs were%eplaced_ with" -other letters oh' certain fixations. 
Thus, use of this information would produce processing difficulties, 
discernible in the eye movement pattern (Onderwoodi & McCbnkie, Note 2) . 

We investigated whether particular aspect., of the visual stimulus pattern 
I present on successive fixations are brought together into a single mental 
representation. Certain characteristics of the ^text pattern were changed 
/during occasional saccades, such as the spacing between, words or the . 
forms of the letters. • If the system Attempts to integrate these aspects 
of the .tjisual array across successive fixations some degree of difficulty 
should be encountered, again being reflected in the eye movement pattern 
(McConkie 4 Zola, 1979? Rayner, McConkie, 4 Zola, 1980). 

, He studied the time "characteristics of the perception and eye movement 
systems by producing changes in the display at specific times following 
the onset of a fixation and observing the effect which this had on the- 

' shapes of the distributions of fixation times (Wolverton, Note 3? • 
McConkie, Underwood, Zola, 4 Wolverton, Note 4). 
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. < r ■ 5* 

These types of studies make atroatf demand* 'both on the ' eye movement 
monitoring equipment and on the equipment used to produce and manipulate the 
stimulus. Decisions in the choice of equipment to purchase or develop 'are 
crucial; wrong choices can greatly limit the research which can be ! done and, 
introduce : undetected ai^tif acts ihto the data, - Computer programming must be 
tight and well controlled. However, the nature of the. constraints on the ^ 
equipment and programming depends on the characteristics of the research to be 
carried out: these constraints are very stronger some types of studies and 
iess strong for others. In order to describe the concerns that must be taken 
into consideration we will discuss two cases in which eye 'movement contingent 
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control of a CRT. display is required, one in which the display must be changed 
during the period of a saccadic .eye movement, and a second in'which it is 
necessary to make a change at some time after the beginning of a fixation. 

* . • " .. . 

Changing t llft ftjunlav during a Saccade' 

. "inorder to change the stimulus display during a* sac cade, there are four 
things that must be accomplished prior to the beginning of 'the following , 
fixation: 1) detecting the oflset of the saccade, 2) determining that this is 
the saccade on which a change is desired, 3) changing the computer's display 
instructions so they can create the new image, and 4) actually having the new 
image present on the CRT. How much time there is to complete all- this depends 
on the length of the saccade on which the change is desired. In .reading, the 
shortest saccades can be completed in less than 20 msec, for instance. If the 
task is to look from one specified point to another which is some distance 
away, this time can be 90 msec or more, depending on the distance of the two 
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points. " o 

Detecting t-h ft n " Het nf> the Sao^ade 

How early in the saccade the eye movement monitoring system can detect 
that the eyes have begun to move depends on several factors. First is the 
speed of throughput of the eye movement monitor. Given that the monitor 
indicates that the eyes are at a given position, how much time has passed 
since they were actually 5 in that position? With the scleral reflection 
approach to eye movement monitoring this throughput time can be a matter of no 
more than three or four msec-or even less, 'unless longer. delays have been 



introduced into the circuitry in order to filter out noise. With some forms 
of filtering, this delay can be over 20. msec. Therefore, in some cases a. 
saccade could be completed. before the eye movement monitor indicates that it 
has begun. With equipment using television technology to photograph the eye, 
and then processing the digitized Image to Identify the eye position, the .eyes 
can begin to move during the 16 msec period required 'to complete one scan of 
the eye. Whether the output shows the eyes beginning to move depends on when 
during the scan the movement began, and whether the critical Information, in 
the image used to track the eyes lies toward the top or bottpm of the 'frame. 
Thus, It would be, quite possible to miss the beginning of the movement in one 
frame, and only detect It in the next, so that a short movement may be 
finished before its Initiation is detected. 



The second factor Influencing how early an eye movement can 'be detected , 
is the rate at which the eye position is sampled.- How much time elapses 
between taking successive samples of the eyes' loca't*<m? At least that. much 
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time can pass after the eyes begin to move before that mdvement isTtteteeWd. 
In our own work we sample the eye position every msec, and" computer aow much - 
movement has occurred in the last M msec. If £his movement is above 
threshold, we then require a oertfen number of additional samples to be above, 
thres'hold, as well. Depending on other factors, it may be possible in this * 
way to reliably detect , the onset of a saccade within,, a relatively few msec of 
the time that the eye movement signal begins to show the movement. 

The third factor is the amount of noise in the eye' movement signal. The. 
noise level essentially defines a region of indeterminacy around the eye's 
position. The, eye movement " aignaTmusOiove^u^ 

movement c&n be reliably detected. If" the noise level is high, then the eyes 
must move further before the movement can be reliably detected.' This 
Increases the likelihood that an eye movement sample will fall to indicate 

■ r ■ . * . V . 

t © i 

' that the eye has begun to move, thus delaying the time until the movement is - 

detected. 

• - - -r-ry • ' 

These three factors are additive. That is, the total -amount of delay 

which can occur in* detecting the onset of a saccade Is at least the total of 
the maximum delay possible from each, of, these three sources. This total delay 
can be sizeable. A combination of slow throughput, -slow sampling rate,- and ' 
high noise level can result in total delays of 50 msec or more* 'In this case, 
all but long saccades "would be completed before their Initiation was detected. 
The combination of fast throughput, high sampling rate- and low noise level can 
permit saccades to be reliably detected within less, than./io'msec- after their 
initiation* ^ A 
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p ft fc«wnintng Whether -a flhfmF* Should -Oocur^ on this Saccate 

The amount of time required for this stage varies widely, depending on 
the requirements of the study. For example, in the simple case; display, 
changes are > called for on every saccade, every leftward saccade, or' on the, 
5th, 10th and . 15th -saccades made. In a- more complex case, a change- is made 
only, if the prior fixation is in a certain region of the display. 

• The most difficult cases .are those where the dlsplay^change is made 
contingent on "aspects of characteristics of the saccade itself. For instance, 
a change may be' desired onty if the saccade will be of at least a. certain 
length, "V will take the eyes to a certain location. These decisions require 
waiting during the saccade to see if it reaches a certain velocity or passes 'a 
certain boundary^ or* until sufficient information is available to" permit an 
.accurate prediction of the location of the following fixation. Obviously, , 
these latter types of decisions require eye movement -data obtained at high 
sampling rates and with low noise, and they leave only the latter part of the 
saccade time available for making stimulus changes prior to the beginning of 
the „ following fixation. 

Matting cha n ts ift the nispiav Instruct i ons ^ 

The 'time required to make changes in the display instructions can also be 

> 

quite variable from study to study. This time consists of the total time 
required to calculate the necessary changes, and, if necessary, to transmit 
those changes (or a cd,py of. the changed list of display oommands) to the 
display device. In the simplest case, alternative Images have been previously 
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^prepared and ai£ present within the computer's high-speed memory. The CPU, or 

•a special' display processor having access to high-speed memory, controls the • 
display device directly^ In this case, when a change in the display is 
required, it is simply a matter of taking display commands from a different 
region of memory. Many display changes can be accomplished in this simple 

'manner. ^On the other hand, if there are many possible forms which the next 
display can takej contingent upon eye position, then" each alternative must' be 
computed when it^is needed. This computation time varies with the complexity 
of the displays invdlyed. Also, once the display commands have been modified, 

„.if they must be transmitted to '-the memory of the display device, there will be 
an additional delay, the amount of which, depends on the speed Of transmission 
of information between the\ devices involved. 

Bgaliaing fcha Tmage on the ftlaolav Device. / 

■* 

The most common electronic displays for psychological research involve 
illuminating the likge on the screen a point- at a time. This is done either 
as a complete raster scan dr in a sequence more 'optimally related to the 
characteristics 'of the display itself,, using a "point-plotting device. In 
either case, the process takes time, ^The'aindunt of time for raster displays;, 
is usually either 16 or 32 msec {.'for pdi'ntrHbt.ii^ t '"e^ip^gt Jthrtime depends 
* on the complexity of -the image and the" efficiency with which it was' coded*. 
However, this does not mean that^thd image .can be realized \n ^he screen 
wlthlnthese period of time afteAhey lire Called for. ; in many" lnstafto.es, it 
is 'not possible to begin' "painting"- a new io-age.on .the screen until the ' 
beginning of a, refresh cycle. That means that with a 16 msec refresh cycle, 
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if a new image were called for just after the beginning of a cycle, it would 
be necessary to complete that cycle and then to display the new image on the 

next cycle. Thus, a total of 31 msec couldNelapse before the initial display 

\ 

of the next image were complete. W^th a 32 msec refresh rate, that time could 
be as much as 63 msec. Thus, substantial delays pan result at this point. 

Two approaches can be taken,, to reducing this source of delay. The first 
is to use simple images with a point-plotting device so that the image can be 
refreshed rapidly. The second approach is to have the facility to begin 
displaying a hew image in mid-cycle. With a point-plotting ^device it is 
possible to. break the cycle at any point and begin displaying an alternative 
image. Thus the change can be completed within the refresh cycle time. Also, 
with some graphics equipment it is possible to point the controller to a new - 
region of high-speed memory at the end of any horizontal scan Juring the 
refresh process. It is therefore possible to display the new image within the 
period of a single refresh cycle after it is requested. 

Finally, some recent raster scan graphics controllers have a d^^L of 
flexibility in the refresh rate, which makes it possible to .attain refresh 
cycles requiring less than the normal 16 msec period. Further development of 
_.__this° equipment could- greatly- facilitate- eye-movement - contingent- display — 



control where complex displays are required. . 
ftummarv " 

"When m^ during the period of a saccadic eye 

movement, there are a number of steps which must^occur within a time ranging 
from -20 to 90 mse-o, depending on th e length of t hjB^saccades, ijw,o^ed_injbhe_ — 
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study. During this period of time it is necessary to detect the onset of the 
saccade, determine whether -a display change should occur during, this saccade, 
make the necessary modifications in the display instructions within" the 
computer and perhaps transmit these to the display device, and actually 
realize the new image on the screen. In order to accomplish this, it is 
necessary to have equipment with the required characteristics and to program 
it with a concern for minimizing delays. 

Desirable system characteristics are the following: 

1. Eye- movement monitoring equipment with fast throughput, low noise, and 
which yields new information with high frequency. 

• ■» i 

2. A program which samples the eye position with a high frequency, which can 
detect, saccade onset and make the decision about whether to Initiate a 
display change as early as the study permits, and which minimizes the 
amount of computation Involved in changing the image. 

3. A display device capable of being rapidly refreshed and of initiating the 
presentation of a new image part way through the refresh cycle. 

How many non-optimal^^ 

depends, of course, on the demands of a given study. 



Changing t h.» THsnlav j V r>^c a Fixation- 
When it is necessary to make an experimental manipulation at a certain 

eft 

time after the beginning of a fixation, many of the concerns described In the 



prior section again apply. This is particularly true if the change must occur 
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relatively early in the fixation, such as 20 to 50 msec after it begins. 
Again, one is faced with the probl^ 

are required in the time which is available. If the change is not required - 
until 100 msec or more after the onset of the fixation, more time is available 
for the steps required. 

However, there is another concern which arises -when dealing with changes 
during a fixation. When manipulating the stimulus during saccades, there is 
less concern about just when the change, takes place. It has been our 
experience that the visual system is quite Insensitive to display changes made 
.while the eyes are moving. Blanking out the display is detected, but 
replacing text strings with other strings is not. thus, it is not critical 
exactly when during the saccade the change is made. . However, when making 
changes during the fixation, the timing of the change often becomes the point 
of the research. In order to make the change at the specific time desired, it 
is necessary to accurately identify the beginning of the fixation, since that 
is the base for timing. There are two problems which arise here, one having 
to do with the equipment being used, and the other with characteristics of the 
eye movements themselves., 
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With respect to the equipment used for monitoring the eye movements, all 
of the concerns described earlier related to detecting of the onset of a 
l,accade"apW^ 

Tf^^^pMMn^'^rthemore, there can be considerable variability in 
how soon after the beginning of a fixation it is detected. For instance, with 
a 16 msec sampling rate there is an Inherent 16 msec variability in when the 
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fixation is said^to "begin. With .the noise in_a-«*latltrtjMl«-«|^^^ 
.variability - can- become- 32-msec.' --W 

increases still further. Variability in determining t^ i j ? intJr m whAch 

timing should begin results in variability in the time that elapses from the 
actual beginning of the fixation until the display change takes place. 



Furthermore, as described earlier, characteristics of the refresh process oan 
add variability to the amount of time that elapses from the time a display 1 
change is called for until it is actually displayed. How much variance in - 
timing can be tolerated depends ,^f ^ 

If relatively precise control is -neede^, equipment for the, research must be 
selected with care. 

The preceding discussion was based on the assumption that an accurate, 
noise-free signal sampled at a high rate would dlearly indicate when a 
fixation begins. Howeyer, this is not the case. As theses decelerate 

during a saccade there is typically a period of overshoot, with the signal 

coming to a peak and then moving back the other direction and gradually 
stabilizing. 'This probably represents a settling time of the eyes, during 
which they center themselves in the socket, and perhaps regain their shape 



"after" responding to^hTl^ces^fnThT ocular muscles. This overshoot is 
exaggerated in equipment which monitors reflections from the lens as well as 
from the eye's surface, suggesting that the torque applied to the eye may 

to -normal at the end of the eye movement. The problem, of course, is what to 
identify as the beginning of the fixation; whether this should be the peak of 
the overshoot at which time the forward component-of^he-movement"is- 
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* completed; whether it should be at the end of the settling period, or whether 



Tt should be at We "other time. There is no clear answer at the present time^ 
ISidl^rada^varia^ and across studies in this area. It is our 

contention that the beginning of-, the fixation should b^idenUfJ L edj*ith_the___ 
point at which visual information of the type needed for the task being used 
can first be acquired from the jdispJ^y^V^y!_sho^ 



"research to attempt to identify where this point occurs. 



-0ne-f£nai-commetit-st^^^ regard to the amount of variability 

- * .-• : : 



: involved in identifying the beginnings of saccades and fixations . In many eye 
movement studies the primary data of Interest are the durations of fixations. 
The duration of a fixation is, of course, simply the time from the end of one 
saccade until the beginning of the next. The error variance in fixation 
durations, then, is a sum of the error' variance In identifying eacn of these 
defining events, since It is reasonable to assume that the two sources of 
variance are uncorrelated. Thus, while the previous discussion has been 
concerned primarily with the problems which this sort of variance produces for 



-employing-eye^ovement-^^^ , irTf act many of the same 



_conc^rns^ifli^evj^ 

where accurate eye movement data are desired. Those factors that contribute •" 
to accurate identification of saccade and fixation beginnings also contribute 
to accuracy in fixation duration data. 



Sgmmapv 



Making display changes at precise times following the onset of a fixation 
squires the' ability^ to reliably detect when the fixation begins. ; This 
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requires equipment which has fast throughput, a low noise level, and which' can 
-be-sampletrat-a- high rat e; Even then, noise~free~ eye movement data would not 
indicate a clear point at which the fixation begins. This adds' another source 



of variance. Research .aimed "at identifying at what point in the' fixation 

perception of visual detail is' possible may help; resolv e this probl em J _____ : _. 

•^v j — -• ■ s *' ; — t ■ 

==— FiS£t^^ monitoring Issues which^ have 'been discussed are not 

only of importance in controlling the stimuli con^nge nt upon eye ' movem ents^ 

^^they are also of concern "if the desire is to obtain an accurate measure of 
fixation times. 

Additional Conoerna \ 



There are- two additional 7 equipment .concerns that should be mentioned. 
First, the necessity of making fast display changes requires that the display 
image fac^ quickly. Thus, for CRT displays a fast-decay. phosphor. is required. 
Second, rapid sampling of the eye position generates a great deal of data, 
requiring large amounts of storage space. It may seem reasonably to^bypjiss__ 
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"this^reqloirement by doing on-line reductlbn-of the data as they are being 
-collected^ — We^b^rievX^^s^to'be^unwlse. Having a complete data record is 
useful for three purposes. If the time at which display changes take place is 
recorded in the data, for Instance by setting a bit pattern In a data word 
collected. at the time a display change is called for, It is possible to verify 
that 'the system was operating properly. n this type of researoh, there is no 
other way to be sure that this is the case. Also, more accurate data 
reduction programs can be developed when reduction' is off-line. There are not 
the time limitations, and the program can move forward or backward along the 
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data stream to find saccade beginnings and endings. Finally, there are 



'frequently irregularities in the' eye movement data, probably resulting from 
blinking and squinting, -These can -lead -to-strange patterns in the redU ^ dN _ x ___„ 
data7and examining the raw ' data' cai^n^cate ^eth^r .n^thesejata^_^_ 

_usable. ' ^^---^ 

Example ' 

In our eye movement contingent display control "system, we use the SRI 
Dual-Purkinje Image Eyetracker (Cornsweet & Crane, 1973). It is claimed to 
have a throughput of about 4 msec, the noise level places a band of 
indeterminacy around the mean slgml value equivalent to leas than 2 min of 
' arc of eye movement. Thus, an eye movement can be reliably detected by the 
time the eyes have moved 5 min of arc or less. We sample 'the eye position . 
every" msec, checking the distance moved over the preceding msec. The peak 
of the overshoot at the end of the saceade la detected by a change in 

direction of the values being obtained over a* 4 m sec period, and the end of 

the overshoot period is detected by finding less than 4 min of arc of movement * 
over a 4 msec period. Furthermore, we have,found that we oan predict the 
location of .the' next' fixation, usually within Hp min of ar 0| once we have 
Identified the point of peak velocity within a saccade., 

The eye movement signal is sampled by a PDP-11/40 computer, which, In 
addition to the CPU, has.a display processor which has access to high-speed 
memory./ The display processor controls a point-plotting CRT. With this, we 
can present a single line of text with a 3 -msec . refresh rate. Thus,, the . 
— entire line can be changed within 3. msec. 
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Most of our studies involve a relatively few alternate lines which can be 



•displayed, contingent upon the eye's location and state. Thus, .the 
alternative lines are typically stored in high-speedmemory J _^^ 



"simply involve the CPU directing the display processor to a new region of 

high-speed memory where an alternative display list resides. Thus, the change 

requires minimal time, with no transmission time required.- 
• 

l In our most recent studyp we were able to detect the onset of a saccade, 
predict the location of the following fixation, and change the line of text if 
the fixation was going to be on a particular word, within the period of all 
but^he-shortest saccades. In this way, it was possible to study the value of 
obtaining perlpneral-vlsual Information from'a word on its later 
Identification. — 

The ability to exert eye movement contingent control over visual ^and__^ 
other stimuli provides a powerful research technique which permits detailed . 
investigation of perception and eye movement control as people are engaged in 
on-going tasks. This technique has been* used in the study of reading and in 
some simpler tasks. It should now be extended to the investigation of visual 
search, picture perception, and other visually-based tasks. 
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